JGram: Rapid Development of Multi-Agent Pipelines for Real-World Tasks

Many real-world tasks can be decomposed into pipelines of sequential operations (where subtasks may themselves be composed of one or more pipelines). JGram is a framework enabling rapid development of such multi-agent systems. Each agent’s services are specified in the JGram Description Language (JDL), and automatically converted into Java source templates. These services may be invoked synchronously (analogous to function call) or asynchronously (analogous to message passing), in a manner that is transparent to the service’s implementation. Complex tasks are created by composing several agent services into hierarchical JGram pipelines in which each agent may dynamically delegate its subtasks to other agents in a recursive manner, and in which errors are handled by a cross-agent, exception mechanism. Although JGram agents communicate using Java’s Remote Method Invocation (RMI) protocol, the framework provides significant enhancements such as authentication, encrypted channels, and dynamic service specification. JGram has been used to develop several real-world agent systems. This paper discusses ARGUS, a visitor identification system that integrates a security camera with face detection, face recognition and user notification systems to automatically identify regular visitors arriving at the front door of our building. 1: Introdu tion Complex real-world problems can often be modeled as collections of simpler, sequential operations. For instance, the Unix environment provides a large collection of relatively simple tools (filters) that can be creatively chained together (using pipes) to solve non-trivial problems. This paper describes JGram, a generalization of the Unix pipe concept as applied to multi-agent systems, where the role of the filters is played by agent services. Aside from the immediate observation that a JGram pipeline is not restricted to a single machine, it differs from a standard Unix pipe in the following three respects. First, the stream oriented communication is replaced by the JGram slate, an arbitrary collection of named, typed entities. Second, agents involved in a JGram pipeline can dynamically alter the pipeline’s structure (itinerary). Finally, unlike a Unix pipe, a JGram pipeline provides extensive support for error handling. Unix pipes are useful mainly because they allow rapid composition of basic functions. In the same spirit, agents in the JGram framework can initiate a JGram pipeline by specifying a sequence of recipients. The JGram framework minimizes the drudgework involved in creating agent services by automatically converting high-level agent specifications (written in the JGram Description Language) into Java source code. These services may be invoked synchronously (analogous to function call) or asynchronously (analogous to message passing), in a manner that is transparent to the service’s implementation. JGram is thus a Java-based agent framework specialized to enable the rapid integration of existing software into a multi-agent pipeline. This paper begins by motivating the usefulness of such a tool by briefly describing a real-world visitor identification system that was built using JGram. The paper then details the various aspects of the agent framework, with each feature being discussed in the context of the visitor identification scenario. An overview of relevant related work is provided, and promising directions for future research are outlined. 2: The Visitor Identi ation Task Consider the following scenario. Visitors to large apartment complexes are typically screened by a security guard in the lobby before being allowed to enter. Over time, guards learn to associate frequent visitors with the tenants whom they plan to visit, and are able to immediately notify the visitor’s host of the guest’s arrival over the building intercom. ARGUS (named after the vigilant watchman from Greek mythology) is an automated version of such a security guard. At a high-level, ARGUS’s operation consists of the following steps, each of which is managed by one or more agents. A security camera photographs the building entrance every two seconds, and a motion detection algorithm identifies potential scenes containing visitors. Faces from these images are extracted using a neural-network-based face detector [13]. A face recognition system, ARENA [15], examines these face images and attempts to find visually similar matches in its stored database of visitors. Any user interested in receiving notification of visitors runs a user-interface agent which is automatically informed when the relevant visitors are identified. This agent also allows users to provide ARGUS with immediate corrections for identification errors. Since ARENA is capable of online learning, this feedback can be immediately incorporated into the recognition dataset. For more information on the ARGUS system, please see [16]. ARGUS is implemented as a collection of agents in a multi-agent system for several reasons. First, the components require different platforms: for instance, the camera interface is limited to Windows, while the face recognition system prefers Linux. Similarly, ARGUS users, distributed over an intranet, require notification on their individual workstations (running either Linux or Windows). JGram agents, which use Java RMI for communication, are well suited for this scenario. Second, the computational load imposed by some of the image processing routines is severe enough to merit splitting the task over multiple machines. Third, a multi-agent architecture offers a high degree of modularity, allowing ARGUS agents to be dynamically added to or removed from the system. For instance, interface agents can be created and killed as users arrive and leave without affecting the rest of the ARGUS system. Similarly, monitoring agents can be inserted to diagnose problems without disrupting service, and different face recognition algorithms can be seamlessly tested. Figure 1 presents an overview of the ARGUS agents and their interactions, and each agent is briefly described below. The delegator agent provides several services, such as an interface to the camera, rudimentary agent name services, and manages the image processing agents which perform the bulk of ARGUS’s work. The delegator creates a JGram pipeline of the form: delegator ! detector ! recognizer ! : : : ! notifier. The detector agent, a wrapper around an existing face detection system, locates human faces in its input image. If a face is not found, the detector signals this failure by throwing